Portable hand-held device for manipulating images

ABSTRACT

A portable hand-held apparatus is provided having a memory for storing images, a motion sensor for detecting motion of the device, and a processor for receiving image manipulation instructions via encoded media. The processor processes the stored images based on the detected device motion and the received image manipulation instructions to produce output images.

CROSS REFERENCES TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.13/021,780 filed on Feb. 6, 2011, issued on Apr. 16, 2013 as U.S. Pat.No. 8,421,869 which is a continuation of U.S. application Ser. No.12/642,831 filed Dec. 20, 2009, issued Mar. 15, 2011 as U.S. Pat. No.7,907,178, which is a continuation of U.S. application Ser. No.11/499,806 filed Aug. 7, 2006, issued Jan. 12, 2010 as U.S. Pat. No.7,646,403, which is a continuation of U.S. application Ser. No.09/113,090 filed on Jul. 10, 1998, issued Sep. 19, 2006 as U.S. Pat. No.7,110,024. Each of the above identified patents and applications ishereby incorporated herein by reference in its entirety. With respect tothe present application any disclaimer of claim scope made in the parentapplication or any predecessor or related application is herebyrescinded. Further, any disclaimer of claim scope that may occur in thepresent application should not be read back into any predecessor orrelated application.

FIELD OF THE INVENTION

The present invention relates to digital image processing and inparticular discloses A Camera System Having Motion Deblurring Means.Further the present invention relates to the field of digital imagecameras and in particular discloses a camera system having motion blurcompensating means.

BACKGROUND OF THE INVENTION

Motion blur in the taking of images is a common significant problem. Themotion blur normally occurs as a result of movement of the camera whiletaking the picture or otherwise as a result of movement of objectswithin an image. As a result of motion blur, it is often the case thatthe image taken is non optimal.

SUMMARY OF THE INVENTION

According to an aspect of the present disclosure, a camera system foroutputting deblurred still images comprises a portable handheld cameradevice comprising an image sensor for recording an image; atwo-dimensional accelerometer for detecting an angular velocity of thecamera system relative to an external environment and to produce anangular velocity output indicative thereof; a linear image sensor forsensing data provided on an encoded card inserted into the camerasystem, the encoded card containing instructions for the manipulation ofthe image; and a processor for receiving the image from the imagesensor, receiving the angular velocity output from the two-dimensionalaccelerometer, and processing the image in accordance with theinstructions sensed from the encoded to deblur any blurred pixelspresent in the image in consideration of the angular velocity output.

BRIEF DESCRIPTION OF THE DRAWINGS

Notwithstanding any other forms which may fall within the scope of thepresent invention, preferred forms of the invention will now bedescribed, by way of example only, with reference to the accompanyingdrawing in which:

FIG. 1 illustrates a schematic implementation of the preferredembodiment; and

FIG. 2 is a schematic block diagram of the main Artcam electroniccomponents.

DETAILED DESCRIPTION

FIG. 2 illustrates a linear image sensor 34 for incorporation in acamera system. The linear image sensor converts Artcard data images toelectrical signals, which are communicated to the ACP. The linear imagesensor 34 is fabricated using either CCD or APS CMOS technology. Theactive length of the linear image sensor is 50 mm, equal to the width ofthe data array on the Artcard.

To satisfy Nyquist's sampling theorem, the resolution of the linearimage sensor is at least twice the highest spatial frequency of theArtcard optical image reaching the linear image sensor. In practice,data detection is easier if the linear image sensor resolution issubstantially above this. A resolution of 4800 dpi (189 dpmm) is chosen,giving a total of 9,450 pixels. This resolution requires a pixel sensorpitch of 5.3 [mu]m. This can readily be achieved by using four staggeredrows of 20 [mu]m pixel sensors.

The linear image sensor is mounted in a special package which includesan LED to illuminate the Artcard via a light-pipe.

The Artcard reader light-pipe can be a molded light-pipe which hasseveral functions:

-   -   1. It diffuses the light from the LED over the width of the card        using total internal reflection facets.    -   2. It focuses the light onto a 16 μm wide strip of the Artcard        using an integrated cylindrical lens.    -   3. It focuses light reflected from the Artcard onto the linear        image sensor pixels using a molded array of microlenses.

The preferred embodiment is preferably implemented through suitableprogramming of a hand held camera device such as that described inAustralian Provisional Patent Application No. PO7991 filed 15 Jul., 1997entitled “Image Processing Method and Apparatus (ART01)”, in addition toAustralian Provisional Patent Application entitled “Image ProcessingMethod and Apparatus (ART01a)” filed concurrently herewith by thepresent applicant, the content of which is hereby specificallyincorporated by cross reference.

The aforementioned patent specifications disclose a camera system,hereinafter known as an “Artcam” type camera, wherein sensed images canbe directly printed out by an internal Artcam portable camera unit.Further, the aforementioned specification discloses means and methodsfor performing various manipulations on images captured by the camerasensing device leading to the production of various effects in anyoutput image. The manipulations are disclosed to be highly flexible innature and can be implemented through the insertion into the Artcam ofcards having encoded thereon various instructions for the manipulationof images, the cards hereinafter being known as “Artcards”. The Artcamfurther has significant onboard processing power by an Artcam CentralProcessor unit (ACP) which is interconnected to a memory device for thestorage of important data and images.

In the preferred embodiment, the Artcam device includes a twodimensional motion sensor. The motion sensor comprises a smallmicro-electro mechanical system (MEMS) device or other suitable deviceable to detect motion in two axes. The motion sensor is preferablymounted on the camera device and its output monitored by the Artcamcentral processor device which is disclosed in the afore-mentionedpatent specifications.

Turning now to FIG. 1, there is illustrated a schematic of the preferredarrangement of the preferred embodiment. The accelerometer 1 outputs tothe Artcard processor 2 which also receives the blurred sensed imagefrom the CCD device. The Artcard processor 2 utilises the accelerometerreadings so as to determine a likely angular velocity of the camera whenthe picture was taken. This velocity factor is then utilised by asuitably programmed Artcard processor 2 to apply a deblurring functionto the blurred sensed image 3 thereby outputting a deblurred outputimage 4. The programming of the Artcard processor 2 so as to perform thedeblurring can utilise standard algorithms known to those skilled in theart of computer programming and digital image restoration. For example,reference is made to the “Selected Papers on Digital Image Restoration”,M. Ibrahim Sezan, Editor, SPIE Milestone series, volume 74, and inparticular the reprinted paper at pages 167-175 thereof. Further,simplified techniques are shown in the “Image Processing Handbook”,second edition, by John C. Russ, published by CRC Press at pages 336-341thereof.

It would be therefore obvious to the person skilled in the art that manydifferent techniques for motion blur removal can be utilised in thepreferred embodiment. Additionally, other forms of motion sensors may beprovided. Once the input image has been deblurred, the image is thenable to be printed out by the Artcam device in accordance with thetechniques as discussed in the afore-mentioned patent specification.

It would be appreciated by a person skilled in the art that numerousvariations and/or modifications may be made to the present invention asshown in the specific embodiment without departing from the spirit orscope of the invention as broadly described. The present embodiment is,therefore, to be considered in all respects to be illustrative and notrestrictive.

The invention claimed is:
 1. A portable handheld device, comprising: amemory that stores images including a first stored image; a motionsensor that generates motion data based on detecting motion of theportable handheld device relative to an external environment in at leasttwo dimensions, the motion data corresponding to a time when the firststored image was captured; a processor; and a light source thatilluminates an image representative of a manipulation to the firststored image, wherein the image representative of a manipulation isconverted to an electrical signal communicated to the processor; whereinthe processor: estimates an angular velocity of the portable handhelddevice at the time when the first stored image was captured, the angularvelocity being estimated using the motion data generated by the motionsensor, receives the electrical signal, and processes the first storedimage to produce an output image based on the angular velocity of theportable handheld device at the time when the first stored image wascaptured and based on the electrical signal.
 2. The portable handhelddevice according to claim 1, wherein the motion sensor detects motion intwo dimensions.
 3. The portable handheld device according to claim 2,wherein the motion sensor is an accelerometer.
 4. The portable handhelddevice according to claim 1, wherein the image representative of themanipulation is received via encoded media.
 5. The portable handhelddevice according to claim 3, wherein the accelerometer comprises amicro-electro mechanical device.
 6. The portable handheld deviceaccording to claim 1, further comprising: an image sensor adapted tocapture a still, blurred image comprising at least one blurred pixel. 7.The portable handheld device according to claim 1, wherein theelectrical signal comprises instructions for manipulation of blurredimages.
 8. A method, comprising: storing an image of a scene including afirst stored image; generating motion data based on detecting motion ofa portable handheld device relative to an external environment in atleast two dimensions by at least one motion sensor, the motion datacorresponding to a time when the first stored image was captured;estimating an angular velocity of the portable handheld device at thetime corresponding to when the when the first stored image was captured,the angular velocity being estimated using the motion data generated bythe at least one motion sensor; illuminating an image representative ofa manipulation to the first stored image; converting the imagerepresentative of the manipulation to an electrical signal; andprocessing the first stored image to remove blur in the first storedimage based on the angular velocity of the portable handheld device atthe time when the first stored image was captured and the electricalsignal.
 9. The method according to claim 8, wherein the at least onemotion sensor detects motion in two dimensions.
 10. The method accordingto claim 8, wherein the at least one motion sensor is an accelerometer.11. The method according to claim 10, wherein the accelerometercomprises a micro-electro mechanical device.
 12. The method according toclaim 8, wherein the image representative of the manipulation is encodedon encoded media.
 13. The method according to claim 8, wherein an imagesensor is adapted to capture the first stored image, which comprises astill, blurred image that includes at least one blurred pixel.
 14. Themethod according to claim 8, wherein the electrical signal comprisesinstructions for manipulation of blurred images.
 15. An apparatuscomprising: a memory; and a processor that: receives image data of ascene from the memory; receives motion data that is based on detectingmotion of the apparatus relative to an external environment in at leasttwo dimensions, the motion data generated by at least one motion sensor;estimates an angular velocity of the apparatus at a time when the imagedata was captured, the angular velocity being estimated using the motiondata generated by the at least one motion sensor; illuminates an imagerepresentative of a manipulation to the image data; converts the imagerepresentative of the manipulation to an electrical signal; andprocesses the image data to remove blur in the image data based on theangular velocity of the apparatus at the time when the image data wascaptured and based on the electrical signal.
 16. The apparatus accordingto claim 15, wherein the at least one motion sensor comprises anaccelerometer.